Reinforcement Learning with Orthonormal Basis Adaptation Based on Activity-Oriented Index Allocation
نویسندگان
چکیده
منابع مشابه
Reinforcement Learning with Orthonormal Basis Adaptation Based on Activity-Oriented Index Allocation
An orthonormal basis adaptation method for function approximation was developed and applied to reinforcement learning with multi-dimensional continuous state space. First, a basis used for linear function approximation of a control function is set to an orthonormal basis. Next, basis elements with small activities are replaced with other candidate elements as learning progresses. As this replac...
متن کاملUnsupervised Basis Function Adaptation for Reinforcement Learning
When using reinforcement learning (RL) algorithms to evaluate a policy it is common, given a large state space, to introduce some form of approximation architecture for the value function (VF). The exact form of this architecture can have a significant effect on the accuracy of the VF estimate, however, and determining a suitable approximation architecture can often be a highly complex task. Co...
متن کاملBasis Adaptation for Sparse Nonlinear Reinforcement Learning
This paper presents a new approach to representation discovery in reinforcement learning (RL) using basis adaptation. We introduce a general framework for basis adaptation as nonlinear separable least-squares value function approximation based on finding Fréchet gradients of an error function using variable projection functionals. We then present a scalable proximal gradientbased approach for b...
متن کاملA Dynamic Allocation Method of Basis Functions in Reinforcement Learning
In this paper, we propose a dynamic allocation method of basis functions, an Allocation/Elimination Gaussian Softmax Basis Function Network (AE-GSBFN), that is used in reinforcement learning. AEGSBFN is a kind of actor-critic method that uses basis functions. This method can treat continuous high-dimensional state spaces, because basis functions required only for learning are dynamically alloca...
متن کاملBasis Function Adaptation in Temporal Difference Reinforcement Learning
We examine methods for on-line optimization of the basis function for temporal difference Reinforcement Learning algorithms. We concentrate on architectures with a linear parameterization of the value function. Our methods optimize the weights of the network while simultaneously adapting the parameters of the basis functions in order to decrease the Bellman approximation error. A gradient-based...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
سال: 2008
ISSN: 0916-8508,1745-1337
DOI: 10.1093/ietfec/e91-a.4.1169